This invention relates to the field of electronic communications. More particularly, a system and methods are provided for reusing a key associated with virtual address space for RDMA (Remote Direct Memory Access) operations. The system and methods are applicable to InfiniBand and other communication infrastructures that support RDMA.
InfiniBand is an input/output (I/O) technology for communicating between host devices (e.g., servers, computing nodes) and target devices (e.g., I/O modules, storage devices, communication links). The InfiniBand architecture defines a point-to-point switched “fabric” for interconnecting hosts and targets, and specifies various verbs, or functions, for establishing and maintaining communication connections between devices. Hosts are coupled to the InfiniBand fabric via Host Channel Adapters (HCA), while targets are coupled to the fabric with Target Channel Adapters (TCA).
InfiniBand technology enables the reassembly of data, by a target, directly into host buffers through RDMA. A host maps local buffers or storage space to virtual address space within the InfiniBand fabric by invoking verbs implemented by an HCA. The host then communicates to the target, out of band, the virtual address space to use for a particular communication connection between the host and the target. A TCA coupled to the target device maintains the state of the connection and performs RDMA to the host buffers via the specified InfiniBand virtual address space.
Verbs specified in the InfiniBand architecture allow a host (via an HCA) to set up a virtual address space for a communication connection and to add or remove host buffers to/from the virtual address space. For example, a host may invoke a first “register” function, while identifying a set of host buffers. In response, an HCA maps InfiniBand virtual address space to the host buffers, and returns an R_Key for identifying the virtual address space. The host may then pass the R_Key to a TCA, and the TCA will use the corresponding virtual address space for a specified connection by producing the R-Key as a ticket to access the space.
Other verb functions (e.g., “re-register”, “de-register”) allow the host to add and remove host buffers to and from a virtual address space. However, each time a host adds or removes buffers to/from an InfiniBand virtual address space, a different R_Key is returned by the HCA. As a result, the host must pass the new R_Key to the TCA, which must update its tables or state information for the communication connection. For applications involving the transfer of significant amounts of data (e.g., streaming media), the extra processing involved in handling constantly changing R_Keys can detract from a device's efficiency and throughput.